A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

Background: Since 2003, the Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents and fellows in its accredited programs. The Resident/Fellow Survey is a screening and compliance tool that programs can also use for continuous quality improvement. However, stakeholders have reported potential problems with the survey's overall quality and credibility. Objective: To redesign the 2006 Resident/Fellow Survey using expert reviews and cognitive interviews. Methods: In 2018-2019, the ACGME redesigned the Resident/Fellow Survey using an iterative validation process: expert reviews (evidence based on content) and cognitive interviews (evidence based on response processes). Expert reviews were conducted by a survey design firm and an ACGME Task Force; cognitive interviews were conducted with a diverse set of 27 residents and fellows. Results: Expert reviews resulted in 21 new survey items to address the ACGME's updated accreditation requirements; these reviews also led to improvements that align the survey items with evidence-informed standards. After these changes were made, cognitive interviews resulted in additional revisions to sensitive items, item order, and response option order, labels, and conceptual spacing. In all, cognitive interviews led to 11 item deletions and other improvements designed to reduce response error. Conclusions: Expert reviews and cognitive interviews were used to redesign the Resident/Fellow Survey. The content of the redesigned survey aligns with the updated accreditation requirements and items are written in accordance with evidence-informed standards. Additionally, cognitive interviews resulted in revisions to the survey that seem to improve respondents' comprehension and willingness to respond to individual survey items.

Probing Liquid-Liquid Phase Separation in Secondary Organic Aerosol Mimicking Solutions Using Articulated Straws.

The presence or absence of liquid-liquid phase separation (LLPS) in aerosol particles containing oxidized organic species and inorganic salts affects particle morphology and influences uptake into, diffusion through, and reactivity within those particles. We report here an accessible method, similar to ice core analyses, using solutions that are relevant for both aerosol chemical systems and aqueous two-phase extraction systems and contain ammonium sulfate and one of eight alcohols (methanol, ethanol, 1-propanol, 2-propanol, 2-butaonol, 3-methyl-2-butanol, 1,2-propanediol, or 1,3-propanediol) frozen in articulated (bendable) straws to probe LLPS. For alcohols with negative octanol-water partitioning coefficient (K OW) values and O/C ratios ≥0.5, no LLPS occurs, while for alcohols with positive K OW values and O/C ratios ≤0.33, phase separation always occurs, both findings consistent with observations using different experimental techniques. When a third species, glyoxal, is added, the glyoxal stays in the aqueous phase, regardless of whether LLPS occurs. When phase separation occurs, the glyoxal forms a strong intermolecular interaction with the sulfate ion, red-shifting the ν3(SO4 2-) peak by 15 cm-1. These results provide evidence of chemical interactions within phase-separated systems that have implications for understanding chemical reactivity within those, and related, systems.

The Positive Effects of Accreditation on Graduate Medical Education Programs in Singapore.

BACKGROUND: There is worldwide interest in assessing the impact of accreditation systems to quantify their benefits to medical education and, through this, health care at the local and global levels. OBJECTIVE: We analyzed ACGME-I Resident Survey data from Singapore for 2011-2018 to assess the impact of accreditation on residents' evaluations of their programs. METHODS: We focused on 7 questions from the annual Resident Survey, which would be affected by accreditation compliance, along with a single global rating of respondents' overall perception of their program. We assessed for differences among specialty groupings (medical, surgical, and hospital-based) and Singapore's 3 health care systems. Repeated measures analysis of variance procedures was used to assess trends across time for the combined 8 items and each individual item. RESULTS: Analysis of the combined items showed significant improvement over the 7 years Singaporean programs had accreditation. There were no effects for specialty type or sponsoring institution. Analyses of individual questions showed 6 of 8 were significant for improvement. For the individual question related to duty hour compliance, there was a significant interaction between time and specialty, suggesting medical specialties showed greater improvement across time compared to surgical and hospital-based specialties. CONCLUSIONS: Implementation of accreditation in Singapore provided educational and clinical learning environment infrastructure not present prior to 2010, with the benefits of this reflected in residents' perceptions of their learning environment. Future assessments of the effects of accreditation might add stakeholder interviews to more fully describe its value and impact.

Relationships Between the ACGME Resident and Faculty Surveys and Program Pass Rates on the ABIM Internal Medicine Certification Examination.

PURPOSE: The Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents since 2003, and faculty since 2012. Surveys are designed to assess program functioning and specify areas for improvement. The purpose of this study was to assess the association of the ACGME's resident and faculty surveys with residency-program-specific performance on the American Board of Internal Medicine (ABIM) certification exam. METHOD: Data were available from residents and faculty in 375 U.S. ACGME-accredited internal medicine programs from the 2012-2013, 2013-2014, and 2014-2015 academic years. Analysis of variance and correlations were used to examine the relationship between noncompliance with ACGME program requirements as assessed by the resident and faculty surveys, and ABIM program pass rates. RESULTS: Noncompliance reported on the resident and faculty surveys was highest for programs not meeting the ACGME program requirement of an 80% pass rate on the ABIM certification examination. This relationship was significant for overall noncompliance, both within the resident (P < .001) and faculty (P < .05) surveys, for many areas within the two surveys (correlations ranged between -.07 and -.25, and P values ranged between .20 and < .001), and for the highest levels of noncompliance across areas of the resident (P < .001) and faculty (P < .04) surveys. CONCLUSIONS: ACGME resident and faculty surveys were significantly associated with ABIM program pass rates, supporting the importance of these surveys within the ACGME's Next Accreditation System.

Chemical Diversity, Biological Activity, and Genetic Aspects of Three Ocotea Species from the Amazon.

Ocotea species present economic importance and biological activities attributed to their essential oils (EOs) and extracts. For this reason, various strategies have been developed for their conservation. The chemical compositions of the essential oils and matK DNA sequences of O. caudata, O. cujumary, and O. caniculata were subjected to comparison with data from O. floribunda, O. veraguensis, and O. whitei, previously reported. The multivariate analysis of chemical composition classified the EOs into two main clusters. Group I was characterized by the presence of α-pinene (9.8-22.5%) and ß-pinene (9.7-21.3%) and it includes O. caudata, O. whitei, and O. floribunda. In group II, the oils of O. cujumary and O. caniculata showed high similarity due amounts of ß-caryophyllene (22.2% and 18.9%, respectively). The EO of O. veraguensis, rich in p-cymene (19.8%), showed minor similarity among all samples. The oils displayed promising antimicrobial and cytotoxic activities against Escherichia coli (minimum inhibitory concentration (MIC) < 19.5 µg·mL-1) and MCF-7 cells (median inhibitory concentration (IC50) ≅ 65.0 µg·mL-1), respectively. The analysis of matK gene displayed a good correlation with the main class of chemical compounds present in the EOs. However, the matK gene data did not show correlation with specific compounds.

Causes of Death of Residents in ACGME-Accredited Programs 2000 Through 2014: Implications for the Learning Environment.

PURPOSE: To systematically study the number of U.S. resident deaths from all causes, including suicide. METHOD: The more than 9,900 programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) annually report the status of residents. The authors aggregated ACGME data on 381,614 residents in training during years 2000 through 2014. Names of residents reported as deceased were submitted to the National Death Index to learn causes of death. Person-year calculations were used to establish resident death rates and compare them with those in the general population. RESULTS: Between 2000 and 2014, 324 individuals (220 men, 104 women) died while in residency. The leading cause of death was neoplastic disease, followed by suicide, accidents, and other diseases. For male residents the leading cause was suicide, and for female residents, malignancies. Resident death rates were lower than in the age- and gender-matched general population. Temporal patterns showed higher rates of death early in residency. Deaths by suicide were higher early in training, and during the first and third quarters of the academic year. There was no upward or downward trend in resident deaths over the 15 years of this study. CONCLUSIONS: Neoplastic disease and suicide were the leading causes of death in residents. Data for death by suicide suggest added risk early in residency and during certain months of the academic year. Providing trainees with a supportive environment and with medical and mental health services is integral to reducing preventable deaths and fostering a healthy physician workforce.

Patterns of Change in ACGME-Accredited Residency Programs and Positions: Implication for the Adequacy of GME Positions and Supply of Physicians in the United States.

BACKGROUND: Recent studies suggest that the supply of primary care physicians and generalist physicians in other specialties may be inadequate to meet the needs of the US population. Data on the numbers and types of physicians-in-training, such as those collected by the Accreditation Council for Graduate Medical Education (ACGME), can be used to help understand variables affecting this supply. OBJECTIVE: We assessed trends in the number and type of medical school graduates entering accredited residencies, and the impact those trends could have on the future physician workforce. METHODS: Since 2004, the ACGME has published annually its data on accredited institutions, programs, and residents to help the graduate medical education community understand major trends in residency education, and to help guide graduate medical education policy. We present key results and trends for the period between academic years 2003-2004 and 2012-2013. RESULTS: The data show that increases in trainees in accredited programs are not uniform across specialties, or the types of medical school from which trainees graduated. In the past 10 years, the growth in residents entering training that culminates in initial board certification ("pipeline" specialties) was 13.0%, the number of trainees entering subspecialty education increased 39.9%. In the past 5 years, there has been a 25.8% increase in the number of osteopathic physicians entering allopathic programs. CONCLUSIONS: These trends portend challenges in absorbing the increasing numbers of allopathic and osteopathic graduates, and US international graduates in accredited programs. The increasing trend in subspecialization appears at odds with the current understanding of the need for generalist physicians.

Effects of potential federal funding cuts on graduate medical education: results of a survey of designated institutional officials.

BACKGROUND: Proposed reductions in federal funding for physician education may affect the United States' ability to produce the number of physicians needed to provide care. OBJECTIVE: Using a survey similar to that used by the ACGME in 2011, we assessed designated institutional officials' (DIOs) perceptions of the impact of potential GME funding reductions. METHOD: In August 2013, we sent a survey link to all DIOs of ACGME-accredited institutions (N  =  678). A 9-item survey asked how future federal funding would affect the number of residency programs in their institutions under 4 different funding scenarios: stable funding, and reductions of 10%, 33%, and 50%. We also asked about changes in the number of residency positions during the last 2 years. RESULTS: The response rate was 47.9% (325 of 678 DIOs); respondents represent 58.9% of accredited institutions with more than 1 program. Most respondents reported no change or an increase under the stable funding scenario. Under a 33% funding reduction, an estimated 17 379 (14.8% of all current) positions would be lost, and a 50% reduction would result in a loss of 33 562 positions (28.6%). Primary care specialties (eg, family medicine, internal medicine) would be most affected under the greatest funding reductions. CONCLUSIONS: The findings of the 2013 survey are consistent with 2011 data, with DIOs projecting significant reductions in programs and positions under more severe budget cuts. DIO comments highlighted reduced optimism (compared to data obtained in 2011) about the effect of funding cuts and concerns about the impact of reductions on patient care and health care personnel at teaching institutions.

Resident operative experience in general surgery, plastic surgery, and urology 5 years after implementation of the ACGME duty hour policy.

OBJECTIVE: Resident duty hour restrictions were implemented in 2002-2003. This study examines changes in resident surgical experience since these restrictions were put into place. METHOD: Operative log data for 3 specialties were examined: general surgery, urology, and plastic surgery. The academic year immediately preceding the duty hour restrictions, 2002-2003, was used as a baseline for comparison to subsequent academic years. Operative log data for graduating residents through 2007-2008 were the primary focus of the analysis. Examination of associated variables that may moderate the relationship between fewer duty hours and surgical volume was also included. RESULTS: Plastic surgery showed no changes in operative volume following duty hour restrictions. Operative volume increased in urology programs. General surgery showed a decrease in volume in some operative categories but an increase in others. Specifically the procedures in vascular, plastic, and thoracic areas showed a consistent decrease. There was no increase in the percentage of programs' graduates falling below minimum requirements. Procedures in pancreas, endocrine, and laparoscopic areas demonstrated an increase in volume. Graduates in larger surgical programs performed fewer procedures than graduates in smaller programs; this was not the case for urology or plastic surgery programs. CONCLUSIONS: The reduction of duty hours has not resulted in an across the board decrease in operative volume. Factors other than duty hour reforms may be responsible for some of the observed findings.

Residents' perspectives on the learning environment: data from the Accreditation Council for Graduate Medical Education resident survey.

PURPOSE: Residents' assessment of their learning environment is an important element of residency accreditation and a strong predictor of resident satisfaction. The authors examined the reliability and validity of a resident/fellow survey and explored the relationship between reported duty hours noncompliance and residents' perceptions of other aspects of their learning environments. METHOD: The Accreditation Council for Graduate Medical Education (ACGME) administered a 29-item Web-based survey in 2007 and 2008 to 91,073 residents in 5,610 programs. Aggregate data from the survey comprised indicators of substantial compliance or noncompliance. The authors examined relationships among duty hours and aspects of the educational environment, as well as the relationship of the survey results to citations from accreditation reviews. RESULTS: The survey demonstrated a high degree of internal reliability (Cronbach alpha, 0.84). Common factor analysis revealed two factors, educational environment and resident duty hours (eigenvalues of 5.49 and 2.42, respectively). Programs having resident-identified duty hours issues were more likely than those without such issues to have received duty hours citations from residency review committees (odds ratio: 2.04; 95% CI: 1.03, 3.05). CONCLUSIONS: The ACGME Resident/Fellow Survey is a reliable, valid, and useful tool for evaluating residency programs. There are strong relationships between duty hours noncompliance and noncompliance in other aspects of the program environment.

Tracking Residents Through Multiple Residency Programs: A Different Approach for Measuring Residents' Rates of Continuing Graduate Medical Education in ACGME-Accredited Programs.

BACKGROUND: Increased focus on the number and type of physicians delivering health care in the United States necessitates a better understanding of changes in graduate medical education (GME). Data collected by the Accreditation Council for Graduate Medical Education (ACGME) allow longitudinal tracking of residents, revealing the number and type of residents who continue GME following completion of an initial residency. We examined trends in the percent of graduates pursuing additional clinical education following graduation from ACGME-accredited pipeline specialty programs (specialties leading to initial board certification). METHODS: Using data collected annually by the ACGME, we tracked residents graduating from ACGME-accredited pipeline specialty programs between academic year (AY) 2002-2003 and AY 2006-2007 and those pursuing additional ACGME-accredited training within 2 years. We examined changes in the number of graduates and the percent of graduates continuing GME by specialty, by type of medical school, and overall. RESULTS: The number of pipeline specialty graduates increased by 1171 (5.3%) between AY 2002-2003 and AY 2006-2007. During the same period, the number of graduates pursuing additional GME increased by 1059 (16.7%). The overall rate of continuing GME increased each year, from 28.5% (6331/22229) in AY 2002-2003 to 31.6% (7390/23400) in AY 2006-2007. Rates differed by specialty and for US medical school graduates (26.4% [3896/14752] in AY 2002-2003 to 31.6% [4718/14941] in AY 2006-2007) versus international medical graduates (35.2% [2118/6023] to 33.8% [2246/6647]). CONCLUSION: The number of graduates and the rate of continuing GME increased from AY 2002-2003 to AY 2006-2007. Our findings show a recent increase in the rate of continued training for US medical school graduates compared to international medical graduates. Our results differ from previously reported rates of subspecialization in the literature. Tracking individual residents through residency and fellowship programs provides a better understanding of residents' pathways to practice.

Residency Programs' Evaluations of the Competencies: Data Provided to the ACGME About Types of Assessments Used by Programs.

BACKGROUND: In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project began to focus on resident performance in the 6 competencies of patient care, medical knowledge, professionalism, practice-based learning and improvement, interpersonal communication skills, and professionalism. Beginning in 2007, the ACGME began collecting information on how programs assess these competencies. This report provides information on the nature and extent of those assessments. METHODS: Using data collected by the ACGME for site visits, we use descriptive statistics and percentages to describe the number and type of methods and assessors accredited programs (n  =  4417) report using to assess the competencies. Observed differences among specialties, methodologies, and assessors are tested with analysis of variance procedures. RESULTS: Almost all (>97%) of programs report assessing all of the competencies and using multiple methods and multiple assessors. Similar assessment methods and evaluator types were consistently used across the 6 competencies. However, there were some differences in the use of patient and family as assessors: Primary care and ambulatory specialties used these to a greater extent than other specialties. CONCLUSION: Residency programs are emphasizing the competencies in their evaluation of residents. Understanding the scope of evaluation methodologies that programs use in resident assessment is important for both the profession and the public, so that together we may monitor continuing improvement in US graduate medical education.

The ACGME Resident Survey Aggregate Reports: An Analysis and Assessment of Overall Program Compliance.

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) uses a 29-question Resident Survey for yearly residency program assessments. This article describes methodology for aggregating Resident Survey data into 5 discrete areas of program performance for use in the accreditation process. This article also describes methodology for setting thresholds that may assist Residency Review Committees in identifying programs with potential compliance problems. METHODS: A team of ACGME staff and Residency Review Committee chairpersons reviewed the survey for content and proposed thresholds (through a modified Angoff procedure) that would indicate problematic program functioning. RESULTS: Interrater agreement was high for the 5 content areas and for the threshold values (percentage of noncompliant residents), indicating that programs above these thresholds may warrant follow-up by the accrediting organization. Comparison of the Angoff procedure and the actual distribution of the data revealed that the Angoff thresholds were extremely similar to 1 standard deviation above the content area mean. CONCLUSION: Data from the ACGME Resident Survey may be aggregated into internally consistent and consensually valid areas that may help Residency Review Committees make more targeted and specific judgments about program compliance.